Miscommunication in Multi-modal Collaboration

We explore grounding and the sub-phenomena of miscommunication and repair from both theoretical and empirical perspectives. From a theoretical perspective, we classify several types of miscommunication, as action or perception failure, and part of a more general case of non-alignment of the mental states of agents. From an empirical perspective, we present a preliminary analysis of examples of miscommu-nication in multi-modal collaboration. These points of view converge towards a predictive model of grounding, which considers costs and beneets of performing grounding acts (including repairs of miscommunication). 1 Overview Our view of miscommunication is that it is just one aspect of the more general phenomena of communication and collaboration. There are a number of complexities to miscommunication, such as what it relates to, who notices and reacts to it and how, which can really only be fully explored within the context of the larger interactive processes One can get a full picture of miscommunica-tion only in contrast to the other issues and alternatives, both alternative interpretations of the communicative state (some of which will be states of miscommunication) and alternative actions that the communicators can take (some of which will be repairs of miscommunication). We explore grounding and the sub-phenomena of mis-communication and repair from both theoretical and empirical perspectives. In the next section, we outline some of the diierent aspects of miscommunication from a formal perspective. In Section 3, we discuss work on the larger issue of grounding in task-oriented collaboration. Then in Section 4, we introduce the multi-modal task-oriented domain that we have used to gather data on actual collaboration. In section 5, we give examples of how the framework developed in the theoretical sections can be used to analyze speciic instances of the observed collaborative behavior. We conclude with some remarks about the potentially beneets of miscommuni-cation, when seen from a pedagogical perspective. 2 What is Miscommunication? Miscommunication is one particular case of a lack of alignment of agents' mental state, speciically one in which they diverge on the occurrence or results of communication. As Austin 1962] noted, communication is action, and thus miscommunication can be viewed as instances of action failure (when the speaker fails to produce the intended eeect), misperception (when the hearer cannot recognize what the speaker intended to communicate), or both. We represent a communicative act as (1), where Done(A;) means agent A performed action and ! means that action has communicative meaning. (1) …